(19) 



0} 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(11) EP 0 920 221 A2 

EUROPEAN PATENT APPLICATION 



(43) 


Date of publication: 


(51) lntCI.6: H04N 9/04 




02.06.1999 Bulletin 1999/22 




(21) 


Application number: 98309537.3 




(22) 


Date of filing: 23.11.1998 




(84) 


Designated Contracting States: 


(72) Inventors: 




AT BE CH CY DE OK ES Fi FR GB GR IE IT U LU 


• Kuwata, Naokl 




MC NL PT SE 


Suwa-shI, Nagano (JP) 




Designated Extension States: 


• NakamI, Yoshlhlro 




AL LT LV MK RO SI 


Suwa-shI, Nagano (JP) 


(30) 


Priority: 25.11.1997 JP 323013/97 


(74) Representative: Sturt, Clifford Mark et al 


28.08.1998 JP 243461/98 


Miller Sturt Kenyon 






9 John Street 


(71) 


Applicant: SEIKO EPSON CORPORATION 


London WC1N 2ES (GB) 




Shinjuku-ku, Tokyo (JP) 





(54) Image processing apparatus and method 

(57) Conventionally, in image processing using a 
smoothing filter, the large amount of calculation for 25 
X the number of pixels takes much processing time. A 
computer 21 as the nucleus of image processing deter- 
mines whether or not a pixel of interest is a color blur 
pixel, based on the Intensity of a low-density element 
color, at steps SI 30 and SI 40. If the pixel is a color blur 
pixel, the computer 21 determines whether or not the 
color blur pixel is an edge pixel at step SI 50. If the pixel 
is not an edge pixel, the computer 21 uses a smoothing 
filter at step SI 52, while if the pixel is an edge pixel, the 
computer 21 uses a median fitter at step SI 54. in image 
processing to reduce the color blur. This reduces the 
amount of calculation and realizes high-speed image 
processing. 
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Description 

Technical Field To which the Invention Belongs 

[0001] The present invention relates to an Innage processing apparatus which performs image processing on Image 
data obtained by utilizing a single-plate solid image pickup device, an image processing method and a medium where 
an image processing control program is stored. 

Related Background Art 



[0002] A digital still camera or the like using a solid image pickup devkse usually employs a single-plate method. As 
shown in Fig. 23(a), in the single-plate method, color filters of R (red). G (green) and B (blue) are arranged In mosaic 
at predetermined ratios, in correspondence with respective pixels of the solid image pickup device. Especially. G color 
filters are zigzag arranged, in a high color component ratio. At each pixel of the solid image pickup device, color signals 
IS of the respective R. G and B colors cannot be obtained, but only one of R, G and B color signals is obtained. Accordingly, 
color signals that cannot be directly obtained at each pixel are obtained by intei^polatlon from color signals of adjacent 
pixels, then all the R, G and B cotor signals are converted into multilevel data. The multilevel data are outputted. and 
display is made based on the multilevel data on a display device or the like. 

[0003] For example, in Fig. 23(a), in the intermediate color filter array indicated by an arrow (->), the left half part 
20 from the center is irradiated with light (white part), while the right half part from the center is not irradiated with light 
(hatched part), as shown in Fig. 23(b). Assuming that the color signal level of respective colors in irradiated stale is 
•1 ■ and that of the same color signals in unirradiated state is "O'. the levels of the R and G cotor signals will have values 
as shown in Fig. 23(c). 

[0004] However, as described above, a G color signal cannot be directly obtained from an R cotor filter, and an R 
2S color signal cannot be directly obtained from a G color filter. Accordingly, the G color signal corresponding to the R 
color filter is obtained from linear interpolation on the G color signal at an adjacent pixel. On the other hand, the R cotor 
signal corresponding to the G color filter is obtained from linear interpolatton on the R color signal at an adjacent pixel. 
Then, the G and R color signal levels have the values as shown in Figs. 23(d) and 23(e). As it is apparent from these 
figures, false color signals occur at pixels around the boundary between an irradiated area and an unirradiated area, 
30 and a color blur occurs on an image by this false color signal. Especially, the cotor blur remarkably appears on the 
boundary between gray and white color areas. Conventionally, to reduce such color blur, a smoothing filter (low-pass 
filter) is applied to cotor difference data of all the pixels constituting image data so as to make the color blur inconspic- 
uous. 

[0005] The conventional technrciue has the following problem. 
35 [0006] The application of the smoothing filter corresponds to diffusion of one cotor component of one pixel to periph- 
eral pixels; that is, matrix calculation is performed by using, e.g., 5X5 pixel matrix having one pixel as the center. For 
example, in the 5 x 5 pixel matrix calculation, calculation is performed 5 x 5 = 25 times on one pixel. Accordingly, in 
case of catoulation on all the pixels, the cateulation amount is enormously 25 x the number of pixels, which increases 
processing time. 



Disclosure of the Invention 



[0007] The present invention has been made in consideration of the above problem, and has its object to provide 
an Image processing apparatus whtoh reduces processing time, upon reduction of color blur in image data obtained 
45 by a single-plate solid image pickup deytoe. an image processing method, and a medium where an image processing 
control program is stored. 

[0008] The foregoing object of the present invention is attained by providing an image processing apparatus which 
performs image processing on image data consisting of dot-matrixed pixels, generated by obtaining image data by a 
single-plate solid image pickup device where a plurality of cotor filters of element color components are arranged in 
so mosato in a nonuniform densities and supplementing the image data by calculation to change the nonuniform densities 
to uniform densities, the apparatus comprising: a color-blur pixel detection unit which detects a cotor blur pixel in the 
image data; and an image processing unit which perfomis image processing on pixels within a predetermined range 
having the detected color blur pixel as a reference pixel, so as to reduce a color blur. 

[0009] The present invention is constructed as above on the premise that image data as an image processing object 
consists of dot-matrixed pixels obtained by a. singe-plate solid image pickup device. In the single-plate solid image 
pickup device, as color filters of a plurality of element cotors are arranged in mosaic in nonuniform densities . the element 
colors are supplemented by calculation so as to obtain uniform densities. However, as described above, a false cotor 
component is generated upon catoulation and a color blur occurs. The cotor-blur pixel detectton unit detects a cotor 
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blur pixel where such color blur occurs, and the image processing unit performs image processing on pixels within a 
predetermined range having the detected color blur pixel as a reference, to reduce the color blur. 
[0010] According to the present invention, in image data obtained by a single-plate solid Image pickup device, a color 
blur pixel is detected, and the color blur is reduced with respect to pixels within a predetermined range having the 
s detected pixel as a reterence. Accordingly, an image processing apparatus which reduces the amount of calculation 
and reduces processing time can be provided. 

[0011] When existence/absence of color blur is detected with respect to a pixel, as the existence/iabsence of color 
blur cannot be uniformly detected from each of element color components of the pixel, the existence/absence of color 
blur is detected from comparison between the pixel and peripheral pixels. Further, as the color filters for the respective 
10 element colors are arranged in nonuniform der^sities in the solid image pickup device, the color blur due to the above- 
described false element color component easily occurs with respect to an element color component of a low-density 
color filter. Accortlingly. it may be arranged such that the color-blur pixel detection unit detects the color blur pixel based 
. on the change rate of element color intensity of a low-density color filter between a detection object pixel and a pixel 
adjacent to the object pixel. 

IS [0012] Accordingly, the color blur pixel can be easily detected by only examining the element color intensity for the 
low-density cotor filter. 

[001 3] As an example of a partteuiar method for detecting a cotor blur pixel, the ootor-blur pixel detection unit may 
detect the color blur pixel based on the change rate of difference between a reference element cotor Intensity and the 
element color intensity for the low-density color filter, between adjacent pixels. 

20 [0014] In this arrangement, the calculatton for color-blur pixel detection can be easily performed only by examining 
the change rate of the difference between the reference element color intensity and the element color intensity for the 
low-density color filter. o ^ 

[0015] For example, as in the intermediate cotor-filter array in Fig. 23 (a), if the alternate arrangement of the R and 
G color filters is known in advance, as the R filters are low-density cotor filters. G cotor is employed as a reference 

2S element color component, and the change rate of the difference ARC = 1R<G I between the reference cotor corpponent 
G and the low-density filter color R is examined between adjacent pixels. The value of the difference ARC is as shown 
in Fig. 23(f). In the change rate of ARC between the adjacent pixels, a pixel around the border between the irradiated 
area and unirradiated area has a value -0.5*. The pixel having the value -0.5" is detected as the cotor blur pixel. 
[001 6] The method for detecting a cotor blur pixel based on the change rate of the difference between element cotor 

30 intensities is not limited to the above-described method. If there are a plurality of tow-density cotor filters, the above- 
described color-blur pixel detection unit may obtain the difference between the element cotor intensities of the tow- 
density color filters, and detect a cotor blur pixel based on the change rate of the difference between adjacent pixels. 
[0017] That Is, as described above, as a false element color component remarkably appears in an element cotor for 
a low-density color filter, the difference between the element cotor intensities for low^lenslty cotor filters is examined. 

35 For example, as shown in Fig. 23(a), if the density of the G cotor filters is high and the densities of the R and B cotor 
filters are low with respect to the entire cotor filter, a false element cotor component remarkably appears in R and G 
colors. In the intermediate cotor filter array, if the B cotor signal levels at the respective pixels are linear-interpolated 
from the adjacent B color filters in the upper color filter array, the B color signal levels at the respective pixels in the 
intermediate color filter array are as shown in Fig. 23(d). As the difference between the R and B cotor signal levels. 

40 ARB = IR-B! = ARG holds. Accordingly, the values of the ARB between the adjacent pixels are as shown in Fig. 23 (f), 
and the change rate of the difference ARB at the pixels around the border between the irradiated.area and the unirra- 
diated area is "b.5V These pixels are detected as the color blur pixels. 

[0018] In this manner, if there are a plurality of low-density cotor filters, the cateulation for color blur pixel detectton 
can be easily made only by uniformly examining the change rate of the difference between component color intensities 

4S tor low-density cotor filters. . • ♦ 

[0019] Whether to examine the difference between a reference element color intensity and a tow-density element 
cotor intensity or to examine the change rate of the difference between low-density element cotor intensities, vanous 
aspects can be provided to examine the change rate between any of adjacent pixels. For example, the change rate 
may be examined between adjacent pixels in eight horizontal, vertical and diagonal directions. It the number of direc- 

50 tions of comparison, is appropriately reduced, the amount of catoulation can be reduced. For the above<Jescribed 
reason, a false element color component occurs at a pixel for a low^ensity cotor filter with a high probability, therefore 
the change rate should preferably be examined between adjacent pixels corresponding to low-density cotor filters. It 
may be arranged such that the cotor-blur pixel detectton unit detects a cotor blur pixel by examining the change rate 
between adjacent pixels corresponding to low-density color filters. 

55 [0020] By this arrangement, as the above-described change rate of the difference is examined between pixels cor- 
responding to low-density cotor filters, the detection can be performed with higher reliability 

[0021] On the other hand, the image processing to reduce a cotor blur can be realized by using a predetermined 
sized smoothing filter, with a detected cotor blur pixel as a reference. As a preferred example of the smoothing process- 
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" Ing. the image processing performs smoothing processing on color difference components obtained by subtracting 
luminance components from element color components of pixels within a predetermined range having a detected color 
blur pixel as a reference, and returns the processed components to the original element color components. 
[0022] That is. the image processing unit performs smoothing processing on pixels within a predetermined range 

s having a detected color blur pixel as a reference. As smoothing on color difference component is effective to reduce 
the color blur, the color difference component, obtained by subtracting luminance component from the element color 
component of each pixel, is smoothing-processed. and returned to the original element color component. 
[0023] In this manner, when a color blur is reduced by smoothing processing, as the color difference component is 
smoothing-processed and returned to the Initial element color component, the calculation can be easily perfomied. 

10 [0024] Further, as the sharpness of an image might be lost by the smoothing processing, it may be arranged such 
that the image processing unit perfomtis edge enhancement processing so as to compensate for the sharpness lost 
by the smoothing processing. . 
[0025] In this manner, the edge enhancement processing can compensate for the lowenng of the image sharpness 

due to the smoothing processing. ..• • 

ts [0026] In execution of edge enhancement processing, if an area where the above-described smoothing processing 
is not perfomied is edge-enhanced, the outside of the smoothing processed area may be unnaturally edge enhanced. 
Accordingly, the imago processing unit may perfomn edge enhancement processing on pixels within a smoothing 
processing range vMere the image sharpness maiy be lost 

[0027] In this manner, as the edge enhancement processing is perfomied on the pwels within the smoothing process- 

20 ing range, unnaturally-Increased sharpness at pixels outside of the smoothing-processing range can be prevented. 
[0028] Upon execution of smoothing processing, it is not advantageous to perform the smoothing processing, uni- 
formly by the same range, on the entire image data. For example, if smoothing processing is performed, by the same 
range, on image data of a large image size and on image data of a small image size, as the ratio of the smoothing 
range with respect to the fomier Image data and that with respect to the latter image data are different, even though 

2S desirable result is obtained from the fonner processing, the entire image may be too vague in the latter processing. 
AccordingV. according to the present invention, it may be arranged such that upon execution of smoothing processing 
with respect to pixels within the predetermined range having the detected color blur pixel as a reference, if the see of 
the image to be processed is large, the image processing unit enlarges the smoothing processing range, while if the 
image size is small, reduces the smoothing processing range. 

30 [0029] In this case, the image processing unit detects the size of Image data as the object of inrrage processing in 
advance, then if the size of the image data Is large, the image processing unit enlarges the smoothing processing 
range, «rhile if the size of the image data is small, the image processing unit reduces the smoothing processing range. 
More specifically, the above operation of the Image processing unit can be realized by storing a plurality of smoothing 
filters of different sizes and selecting one of the filters in accordance with the Image size. 

35 [00301 Accordingly, as the smoothing processing range can be changed in accordance with the size of image to be 
processed, smoothing processing can be performed with an optimum range. 

[0031] Further, as one example of reduction of color blur, the image processing unit detects color difference compo- 
nents obtained by subtracting luminance components from element color components of pixels within a predetetmined 
range having a detected color blur pixel as a reference, then replaces the color difference component having a central 
40 value with the color difference component of the color blur pixel, and returns the color difference components to the 
original element color components. That is. the color blur is reduced by replacing the color difference component of 
the color blur pixel with that of a peripheral pixel. Note that a particular constniction to select a central value in cotor 
difference components of pixels within a predetennlned range may be realized by a predetemiined median filter or the 
like 

45 [0032] In this manner, the central value of color difference components of pixels within a predetermined range is 
used as a color blur pixel, thus the color blur is reduced. . ... ^ . ^ ho,.,,,^ 

r00331 As described above, the smoothing processing is performed on pixels within a predetermined range having 
a color blur pixel as a reference, then the subsequent edge enhancement processing compensates for sharpness lost 
by the smoothing. However, empirically, if color blur pixels exist on an edge portion as a border of an image, the image 

so of the edge portion may become vague. In view of this inconvenience, it may be arranged such that the image Process'ng 
unit detemiines whether or not the color blur pixel Is an edge pixel. If the color blur pixel is an edge pixel, the image 
processing unit replaces the color difference component having the central value, among color difference coniP°"ents 
Obtained by subtracting luminance components from element color components of pixels within a P^^de^rmin^ range 
having the color blur pixel as a reference, with the color difference component of the color bjur pixel. On the other hand. 

55 if the color blur pixel is not an edge pixel, the image processing unit perfonns smoothing processing on tje ^tor 
difference components, obtained by subtraction of luminance components from element ^otor components olpixes 
within the predetermined range having the color blur pixel as a reference, and returns the color difference components 
to the initial element color components. 
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[0034] In this case. If a color blur pixel is detected, the image processing unit determines whether or not the color 
blur pixel is an edge pixel. If the image processing unit determines that the color blur pixel is an edge pixel, the image 
processing unit detects color difference components of pixels within a predetermined range having the color blur pixel 
as a reference then replaces the color difference component having the central value with the color difference com- 
5 ponent of the color blur pixel, and returns the color difference components to the initial element color components as 
described above. On the other hand, if the image processing unit determines that the color blur pixel is not an edge 
pixel, performs smoothing processing on the color difference components of pixels within the predetermined range 
having the color blur pixel as a reference, and returns the color difference components to the initial element color 

10 [(^q^^Fo^r example, if a color blur pixel exists on the border between white and black areas, the color difference 
component of the color blur pixel is replaced with the color difference component of a peripheral pixel, i.e.. the color 
blur pixel becomes a pure white or black pixel, which removes the color blur while prevents vagueness of the border 
portion Further, when it is determined whether or not a color blur pixel is an edge pixel, the determination can be made, 
e q by comparing the change rate of the color difference component with a predetermined threshold value, since the 
change rate of color difference component between adjacent pixels increases at edge pixels. If the change rate of the 
color difference component is larger than the threshold value, it is determined that the cokjr blur pixel is an edge pixoL 
It may be arranged such that an edge pixel is determined based on the change rate of luminance component between 
adjacent pixels, on the assumption that the edge pixel has a steep luminance slope. 

[0036] in this manner, if the cotar blur pixel is an edge pixel, the color difference component having the central value 
within the predetermined pixel range is applied to the color blur pbcel. while if the color blur pixel is not an edge pixel, 
smoothing processing is performed. As a result of processing, the image edge portion IS not vague. 
[0037] The method to detect color blur pixel and reduce the cotor blur with respect to pixels within a predetemfiined 
range having the color blur pixel as a reference is not necessarily realized by a substantial apparatus, but is also 
effectively realized by a methiod. . ■ , ^ \, ^ 

25 [00381 As described above, the image processing apparatus which detects a color blur pixel and reduces a color 
blur with respect to pixels within a predetermined range having the cotor blur pixel as a reference may be a single 
device or may be installed into another apparatus. Thus, the present invention maybe utilized in various aspects. 
Further the present invention can be appropriately changed; for example, it can be realized by hardware or software. 
[0039] In a case where the idea of the present invention is embodied as software to control an image processing 
30 apparatus, the idea of the present invention exists, and can be utilized on a storage medium containing the software. 
The storage medium may be a magnetic storage medium or opto-magnetic storage medium, or any storage medium 
developed in the future; further, duplications from such storage medium such as a primary duplfcate. a secondary 
dupltoate and the like must be equivalent to the initial storage medium. Furthermore, even if a communication line is 
used as unit for supplying the present invention, or even if the method of the present invention is written into a semi- 
35 conductor chip, the present invention can be Utilized. . . ^- .. 

[00401 Further, the idea of the present invention does not change even if it is realized partially by software and partially 
by hardware, and further, partially stored on a storage medium and appropriately read out from the storage medium 
in accordance with necessity. 

40 . Brief Description of the Drawings 

[0041] 

Fig. 1 is a block diagram showing an image processing system to which an image processing apparatus according 
45 to an embodiment of the present invention is applied; 

Fig. 2 is a bkjck diagram showing a hardware construction of the image processing apparatus; 

Fig 3 is a block diagram showing a simple hardware construction of a digital still camera 

Fig. 4 is a schematic diagram showing another application example of the image processing apparatus of the 

so R^risTsch^tic diagram showing another application example of the .mage processing apparatus of the 

present invention; .^.^ocioinn 
Fig. 6 is a ftowchart showing determination of color blur pixel and image processing by the image processing 

apparatus of the present invention; .„^hin« oraa- 

Figs 7(a) and 7(b) are schematfc diagrams showing the relation between an image size and a smoothing area. 
55 Figs. 8(a) to 8(c) are diagrams showing 3 x 3. 5 x 5 and 7 x 7 pixel smoothing filters; „..„Kar 

Figs. 9(a) and 9(b) are diagrams showing the relation between height and width of image data of the same number 

of pixels; , . ■ . . 

Fig. 10 is an explanatory view showing the movement of a pixel of interest in processing, 
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Fig. 11 is an explanatory view showing a case where the change rate of element color component is obtained from 
a difference value between diagonally adjacent pixels; 

Fig. 12 is an explanatory view showing a case where the change rate of an image is obtained from a difference 
value between diagonally adjacent pixels; 
s Fig. 13 is an explanatory view showing a case where the change rate of an image is obtained between all the 

jadjacent pixels; 

Fig. 14 is an explanatory view showing a modification of the case where the change rate of the image is obtained 
between adjacent pixels; 

Fig. 15 is an explanatory view showing another rrradlfication of the case where the change rate of the Image is 
10 obtained between adjacent pixels; 

Fig. 16 is an explanatory view showing an example of color difference component values of pixels in image data 
of an edge portion including a color blur pixel; 

Fig. 17 is an explanatory view showing an example of calculation processing by using a median filter; 
Figs. 18(a) to 18(c) are diagrams showing 3 X 3. 5 X 5 and 7X7 pixel unsharp masks; 
IS Fig. 1 9 is a schematic diagram showing the relation between a smoothing area and an edge enhancement area; 

Fig. 20 is a table showing various parameter values in a one-dimensional simulation model in a case where edge 
enhancement processing is not performed; 

Fig. 21 is a table showing various parameter values in the one-dimensional simulation model in a case where edge 
. enhancement processing is performed by using an unsharp jpnask of the same size/as that of a smoothing filter; 
20 Fig. 22 is a table showing various parameter values in the one-dimenstonal simulation model in a case where edge 

enhancement processing is performed by using an unsharp mask of a size smallerthan that of the smoothing filter; 
and 

Figs. 23(a) to 23(f) are explanatory views showing the prnciple of occurrence of color blur. 
ss Descriptton of Special Embodiment 

[0042] A referred embodiment of the present invention will now be described In detail in accordance with the accom- 
panying drawings. 

[0043] Fig. 1 is a block diagram showing an image processing system to which an Image processing apparatus 
30 according to an embodiment of the present inventkxi is applied. Fig. 2 is a block diagram showing a particular hardware 
construction of the apparatus. 

[0044] In Fig. 1 . an image input device 10 outputs an image obtained by utilizing a single-plate CCD or the like, as 
image data represented by dot-matrbced pixels, to an image processing apparatus 20. By the above-described principle, 
the image data obtained by the single-plate CCD includes a color blur. The image processing apparatus 20 detects a 
35 color blur pixel in the Input image data, then performs image processing within a predetermined range having the color 
blur pixel as a reference so as to reduce the color blur, and outputs the processed data to an image output device 30. 
The image output device 30 outputs the image processed data in dot-matrixed pixels. 

[0045] The image processing apparatus 20 comprises color-blur pixel detection unit for detecting a color blur pixel, 
and image processing unit for performing image processing to reduce the color blur with the detected color blur pixel 
40 as a reference. 

[0046] The image input device 10 is realized by. e.g.. in Fig. 2, a film scanner 11 a digital still camera 12 or a video 
camera 14 having a single-plate CCD. For example. Fig. 3 is a schematic diagram showing a schematic hardware 
construction of the digital still camera 12. In Fig. 3. incident light passes through an optteal system 12a and enters a 
CCD 12b. The CCD 12b has R. G and B color filters arranged at predetermined ratios in correspondence with the 
45 respective pixels. A driver 12c outputs color signals at each pixel. The output color signals are digital-converted and 
inputted into an interpolation calculator 1 2d. The interpolation calculator 1 2d obtains a color signal of an element color 
component, which cannot be directly obtained with respect to a pixel, by linear interpolation calculation from a peripheral 
pixel. The color signals including the signal obtained by the linear interpolation are stored as RGB multilevel data into 
an image memory 12e. 

so [0047] On the other hand, the image processing apparatus 20 is realized by, e.g., a computer system comprising a 
computer 21 . a hard disk 22, a keyboard 23, a CD-ROM drive 24. a floppy disk drive 25 and a modem 26. The image 
output device 30 is realized by. e.g.. a printer 31 or a display 32. Note that the modem 26 is connected to a public 
communication line, and connected to an external network via the public communication line, to download and introduce 
software and data. ^ - 

S5 [0048] In the present embodiment, the film scanner 11 or the digital still camera 12 as the Image input device 10 
outputs RGB multilevel data as image data, and the printer 31 as the image output device 30 inputs CMY (cyan, 
magenta, yellow) or CMYK (CMY + black (K)) binary data, while the display 32 Inputs RGB multilevel data. 
[0049] On the other hand, an operating system 21a operates in the computer 21. and a printer driver 21b and a 



6 



EP O 920 221 A2 



display driver 21c corresponding to the printer 31 and the display 32 are Installed in the computer 21 Further, an image, 
processing application program 21d performs processing under the control of the operating system 21a. The image 
processing application program 21 d performs predetemnined image processing In cooperation with the printer driver 
21b and the display driver 21c. in accordance with necessity Accordingly, the computer 21 as the Image processing 
s apparatus particularly plays roles to input RGB multilevel data, generate RGB multilevel data where image processing 
has been performed to detect a color blur pixel and reduce the color blur, display the data on the display 32 via the 
display driver 21c. and at the same time, convert the data into CMY (or CMYK) binary data via the printer driver 21b, 
and cause the printer 31 to print-output the converted data. 

[0050] Thus, in the present embodiment, a computer system for image processing is installed between the image 
10 input and output devices, however, such computer system is not necessarily provided. The present inventton Is appli- 
cable to a system to perform image processing to reduce a color blur with respect to image data including the color 
blur, resulted from image-pickup by a single-plate CCD and interpolation calculation as described above. For example, 
as shown In Fig. 4, the present invention may be applied to a system where an image processing apparatus to perform 
image processing is installed into a digital still camera 13a having a single-plate CCD. A display 32a perfomis display 
IS or a printer 31a performs printing by using image data where a color blur is reduced. Further, as shown in Fig, 5, the 
system may be constructed such that a printer 31b which inputs image data without computer system and perform 
printing, performs image processing to image data, inputted via a film scanner lib or a digital still camera 13b having 
a single-ptate CCD, to reduce a color blur. 
. [0051] The above-described image processing to detect a color blur pixel and to reduce a color blur is executed by 
20 an image processing program, corresponding to a flowchart of Fig. 6, in the above computer 21 . 

[0052] The method for reducing a color blur employed in the present embodiment uses a smoothing filter as descnbed 
later, similarly to the conventional method, however, it is different from the conventional method in that the range where 
the smoothing filter is used is limited to periphery of a color blur pixel. For example, smoothing processing on two 
image sizes of bit maps as image data to be processed, as shown in Figs. 7(a) and 7(b). is considered. The image 
2S data are processed with smoothing filters of the same size, as represented by hatched portions in the figures. Note 
that If each pixel of the image data one-toone corresponds to each pixel of a CCD. the irmge size differs in accordance 
with the number of pixels of the CCD; further, even if the number of pixels of the CCDs are the same, the image size 
Is appropriately enlarged/reduced. The image data in this case have different sizes from these reasons. 
[0053] As apparent from these figures, as long as smoothing filters of the same size are used, the ratio of the smooth- 
30 ing process area with respect to the entire image changes as the image size changes. Accordingly if smoothing filters 
uniformly of the same size are applied to image data of various image sizes, an appropriate sized area may be 
smoothed, as shown in Fig. 7(a), while sometimes the ratio of the smoothing process area with respect to the entire 
image may be large, as shown in Fig. 7(b). Since smoothing processing is blurring an image, there is a possibility that 
almost whole image is blurred as shown in Fig. 7(b). ^ 
35 [0054] Accordingly, the present embodiment employs smoothing filters of different sizes of, e.g.. 3 x 3. 5 x 5 and 7 
X 7. as shown in Figs. 8(a) to 8(c), and appropriately selects the smoothing filter in accordance with an image size. 
Upon determination of the size of a bitmap Image, (height) X (width) of the image may be calculated to obtain the 
number of pixels, and the obtained number of pixels may be used as an index. However, in the present embodiment, 
the image size of a processing object image is determined in accordance with A represented by the following equation: 



40 



45 



A = min(height. width) (1) 
Note that min(height, width) means smaller one of the height and the width. At step S110. 



if A < 300 holds, 

the 3 X 3 pixel smoothing filter is selected; 
if 300^ A ^600 holds. 

the 5 X 5 pixel smoothing filter is selected; and 
50 if A > 600 holds, 

the 7 X 7 pixel smoothing filter is selected. 



The types of smoothing filters used at steps SI 22 to 81 26 are stored into a work area. The work area may be a RAM 
in the computer 21 or the hard disk 22. ^ * *u • * 

55 [0055] The mtn(height. width) is used as a reference from the foltowing reason. That is, whether or not the size of a 
smoothing filter is appropriate may be determined based on the ratio of the number of smoothed pixels with respect 
to the number of all the pixels. However, there are varfous image data, even having the same number of pixels, such 
as image data having a width longer than its length, as shown in Fig. 9(a). and a typical rectangular image data having 
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a width a little longer than its length as shown in Fig. 9(b). In a case where a smoothing filter is determined in accordance 
with the number of pixels of processing object image, the same smoothing fitter is selected for both image data in Figs. 
9{a) and 9(b) Then the former image is smoothed almost over the whole length in height, although the latter image 
is smoothed with an appropriate smoothing filter. In this case, the former image provides visually blurred impression. 
Accoixlingly. this inconvenience can be avoided by using smaller one of the "height" and "width" of image as a reference. 
[0056] Referring to the flowchart of Fig. 6. in image data comprising dot-matrixed pixels as shown in Fig. 10. it is 
determined with respect to each pixel whether or not the pixel is a color blur pixel, white the pixel of interest is horizontally 
moved in a main-scanning direction, and moved in a subscanning direction. 

lOKT] If the image data comprises dot-matrixed pixels, each pixel is represented by RGB multilevel data ( 0 to 
■255" level) The types of the smoothing filters to be used are stored at steps S122 to S126. then, at step S130, the 
difference ARB of R and B multilevel data at a pixel of interest and its peripheral pixel. The difference ARB is represented 



as: 



ARB=IR-BI (2) 

[0058] As shown in Fig. 23(a), the CCD <rf the present embodiment has an arrangement where the R, G and B color 
filters are arranged in mosaic in correspondence with respective pixels, and only the G color filters are zigzag arranged 
in a high density, while the R and B color filters are arranged in low densities. At a pixel, a color signal which cannot 
be directly obtained is obtained by linear interpolation calculation from a color signal of an adjacent pixel, then the 
obtained the R G and B color signals are converted into multilevel data. Accordingly, as a result of linear interpolation 
calculation at some pixel, lack of initial intensity in low-density R, B multilevel data, or generation of false element 
color component which has not been initially unnecessary, causes a color blur, at a certain probability. 
[00591 Further, it is known that a color blur remarkably appears around the border especially between white and 
black areas. For example, when white color is represented at a pixel corresponding toan R color filter, the RGB mul- 
tilevel data is initially (R, G, B) = (255. 255, 255), however, linear interpolation calculation may result in. e.g.. multiteyel 
data (R G B) = (255 255. 1 27). As the color filters are arranged in a high density, there is a low probability that a false 
G color'colT,ponent ci»urs from interpolation calculation; the R component, that can be usually directly obtained from 
the G color filter, does not include a false color component Initially, to represent white at a pixel, the value of the 
difference ARB is "0". and to represent black, the value of the difference ARB is also "0". However, at a color blur pixel, 
the value of the difference ARB tends to be greater than *0". „ « u 

[0060] In some image data, a pixel where the R and G components have high intensities while the B component has 
a tow intensity exists. However, in this case, the respective element cok^r components are maintained at its adjacent 
pixel and the ARB value gradually changes as it becomes distant from the initial pixel. On the other hand, as a cotor 
blur basically occurs in pixel units, and one of adjacent pixels is white or black or the like, it cannot be wrong to regard 
a pixel having a high ARB change rate between adjacent pixels as a color blur pixel. Accordingly, m the present em- 
bodiment, the change rate of the ARB value between a pixel of interest and its peripheral pixel is examined as described 
later, and a pixel having a high change rate is detected as a color blur pixel. As there is a low probability that the G 
color filters, arranged in a high density, cause a false element color component by interpolatwn cateulation. the equation 
(2) may be replaced with the following equations, using this element color component as a reference; , 

AGR = IG-Rl (3) 
AGB = IG-Bl 

[00611 Further, the examination of the ARB change rate can be preferably made by comparing the ARB change rate 
between pixels corresponding to high^lensity cotor filters. That is. as a false element cotor component f requently occurs 
at a pixel corresponding to a low density cotor filter in comparison with a pixel corresponding to a high density color 
filter, the comparison of the ARB change rate between pixels corresponding to low density color filters enables detection 

Ssai^Actu'aTcotor-blur pixel detection is performed at step S140. where it is determined whether or not a pixel of 
merest is a color blur pixel based on the change rate of the difference ARB. obtained as above, between P^"J^^^\ 
pixels. At this time, as shown in Fig. 11, a matrbc having x pixels in a horizontal direction and y pixels in a vertical 
direction is consklered with the pixel of Interest as a central pixel. 
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E = 4ARB{x, y)-2ARB(x-1,y-1)-2ARB(x+1.y-1) (5) 

Then the value of E and a predetermined threshold value Th Is compared. If E ^ Th holds, It is determined that the 
5 pixel of interest is a color blur pixel. Accordingly, at the above-described step SI 30, the difference ARB at the pixel of 
interest is obtained, and ARB(x-1 . y-1 ) and ARB(x+1 . y-1 ) are obtained as ARB values of peripheral pixels. 
ro0631 The meaning of the detection of the color blur pixel in accordance with the determination reference E > Tn is 
as follows That is. if a color blur has occurred at a pixel of interest, as the 6BB value change rate between the pixel 
of interest and its peripheral pixel becomes high, the value E in the equation (5) is large. If the value E exceeds ttie 
10 threshold value Th. it is detemiined that the pixel of interest is a color blur pixel. Further, referring to the equation (5) 
and Fig 11 the ARB value change rate is examined between the pixel of interest and its diagonally adjacent pixel. 
Referring to Fig 23(a). if the pixel of interest corresponds to R. the orthogonally adjacent pixel, to B; If the pixel of 
interest corresponds to B. the orthogonally adjacent pixel, to R; if the pixel of interest corresponds to G. the orthogonally 
adjacent pixel to G That is, if the pixel of interest corresponds to a low-density element color component, the change 
rate of the ARB value is examined between the pixel of interest and an adjacent pixel corresponding to a low-density 
element color component, while if the pixel of interest corresponds to a high-density element color component the 
change rate of the ARB value is examined between the pixel of interest and an adjacent pixel corresponding to a high- 
density element color component. ^,»,«„ftK» 

r00641 Accordingly, at stops S130 and S140. the color-blur pixef detection is made based on the change rate of the 
difference between element coior iritensities corresponding to low-density color filters. Color-blur pixel detection unit 
is constructed by hardware and software to execute the above detection. 
[00651 If it is determined at step S140 that the pixel of interest is a color blur pixel, it is detemnined whether or not 
the pixel is an edge pixel at step S150. As the determination of edge pixel is eflectively made based on color difference 
component, color difference components CI and C2 are obtained by subtracting a luminance component Y from R 
zs and B multilevel data, in the present embodiment, the color ditference.components CI and C2 are represented as: 

CI = R - Y (®) 

^ C2 = B-Y (7) 

[00661 AS RGB multilevel data do not directly have luminance values, to obtain the luminance, the RGB multilevel 
data rriay be color-converted to Luv color space. However, this is not advantageous due to the problem of the amount 
35 of calculation. For this reason, the following conversion to obtain luminance from RGB. utilized in television, is em- 
ployed. 

Y=0.30R + 0.590+0.116 

40 ■■ 

" [0067] As an edge pixel is a part of border portion of an image, the change rate of the color difference compaients 
CI arid C2 are high between adjacent pixels. If any one of the following two detemiination references is satisfied, a 
pixel of interest is determined to be an edge pixel. 

IC1(x.y)-C1(x-1.y-1)l2Thl (9) 



50 



55 



IC2(x, y)-C2(X-1. y-1)l ;a Th2 



(10) 



(00681 Notethafx-representsahorizontalcoordinate, and v. a vertical coordinate. . ^ . , . . ^ . , 
0069] That is. the expressions (9) and (1 0) mean that, in image data consisting of dotHTiatnxed P'xels having a c<^r 
SuTpLl as a central pbcel as shown in Fig. 12. the change rates of the color dWerence '^^^^^'^^^^'^ ^^^"^.^^^^l 
obtained between orthogonally adjacent pixels, and it is respectively detemiined whether or not the ''.^'^^d^Kerence 
components are greatenhan the threshold values Th1 and Th2. If one of the determination l^^'^'^^^^.f^^^^^^ 
p?xel of interest is detemiined to be an edge pixel. As the pixels are arranged in matrix as shown in Fig 1 3- w^^" 
me central pixel is treated as a pixel of interest, there are eight adjacent pncels. Si^1''-;'y^";« ^^^^.^^^^^^^ 
difference components CI and C2 obtained between adjacent pixels, and compared with threshold values Thi and 
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Th2. If the color difference components CI or C2 is equal to or greater than the threshold value Thi or Th2, it may be 
determined ttiat the pixel of interest is a color blur pixel. 

[00701 In this manner, the determination as to whether or not the color blur pixel Is an edge pixel is effectively made 
based on the change rates of the color difference components C 1 and C2. However, for one color blur pixel, comparison 

5 using the color difference components CI and C2 are respectively performed with respect to eight adjacent pixels, the 
amount of calculation is too large, which might lower the processing speed. Accordingly, the comparison calculation 
may be performed with respect to four pixels in vertical and horizontal directions as shown in Fig. 1 4, or the comparison 
calculation may be performed with respect to diagonally adjacent four pixels as shown in Fig. 15. These methods 
reduces the amount of calculation. 

10 [0071] Further, the determination as to whether the color blur pixel is an edge pixel may be made based on the slope 
of luminance. In this case, the above-described expressions (9) and (10) are replaced wnth the following expression: 



15 



20 



IY(x.y)-Y(x-1.y-1)12Th3 (11) 

[0072] By using the expression (1 1 ) as the edge pixel determination reference, the amount of calculation is reduced 
to 1/2 of that in use of the above expressions (9) and (10). 

[0073] If it is determined at step SI 50 that the color blur pixel is not an edge pixel, the process proceeds to step 
S152 at which smoothing processing is pertornrwd using the smoothing filter detemiined at steps S122 to S126 with 
respect to pixels within a predetermined range having the color blur pixel as a reference. Upon execution of smoothing 
processing, as smoothing of color difference component is- effective, the smoothing processing is performed on the 
color difference components CI and C2, calculated in accordance with the above equations (6) and (7), in the present 
embodiment. 

[0074] Next, the smoothing processing using the smoothing fitter will be described. In each of the smoothing filters 

2S as shown in Figs. 8(a) to 8(c), the central value is used as a weight on the color difference components CI and C2 of 
a pixel of interest in the matrix image data. The value is used for integration with respect to peripheral pixels of the 
pixel of interest with weighting corresponding to values of the cells of the smoothing filter. In this case, every cell has 
a value "1 " therefore, a smoothed color difference component CI ' is obtained by adding up the color difference com- 
ponents CI of the respective cells and dividing by the number of the cells; a smoothed color difference component C2* 

30 is similarly obtained. The respective cells may have appropriate weights respectively However, the use of smoothing 
filter means execution of the above-described matrix calculation, and therefore, if the respective cells have weights, 
the amount of calculation increases. That is. in the present embodiment, as the respective cells of the smoothing filters 
have the value "1 the data of the respective pixels may be added up and divided by the number of cells. However, tn 
a case where the respective celts have weights, multiplication and addition are required for the number of cells, which 

35 increases the amount Of calculation. 

[007S] If such matrix calculation is performed with respect to all the pixels of innage data, as in the conventional 
processing, the huge amount of calculation requires much processing time. AccordingV. the processing of the present 
embodiment to detect a color blur pixel and perform smoothing around the color blur pixel is expected to greatly reduce 
the amount of calculation, and enable high-speed image processing. 

40 [0076] On the other hand, if it is determined at step SI 50 that the color blur pixel is an edge pixel, the process 
proceeds to step S154 at which a 5 X 5 pixel median filter is applied to pixels having the color blur pixel as the ceritral 
pixel Note that the size of actualV-used median filter is not necessarily 5X5 pixel size, but may be 3 x 3 pixel size. 
Thus, the size of the median filter may be appropriately selected. Then, for convenience of explanation, a case using 
a 3 X 3 pixel median filter will be described. . . ^ » , 

4S [0077] For example, as shown in Fig. 16, a 3 X 3 pixel dot matrix having a color blur pixel as the central pixel is 
considered. Note that the values of the respective cells are values of the color difference components CI . The diagonal 
line in Fig 16 corresponds to an edge direction. Using the 3 x 3 pixel median filter means sorting the va ues of the 
color difference components CI of the all nine pixels in ascending or descending order, selecting the central value and 
replacing the value with the value of the color difference component CI of the color blur pixel. That is, in Fig 16, six 

so pixels have -lO- CI values, one pixel has •40" CI value, and three pixels have -lOO- CI values, f these va ues are 
sorted in ascending order, the result is as shown in Fig. 17. It is apparent from Fig. 17. the centraf value is 10 . and 
the value of the color difference component CI of the color blur pixel is MO". rtiff«rflnra 
[0078] On the other hand, if the 3 x 3 pixel smoothing filter is used, the value of the smoothed color d Werence 
component CV is •33^ obtained from dividing the total value "SOO- of the cell values by the number of pixels 9 ^ As 
55 the smoothed color difference component CV is obtained by averaging and adding up the color driference ^ornponen s 
CI of the peripheral pixels of the color blur pixel, the image data is smoothed. The result of smoothing ^ eq^-vate^t 
the result of so-called low-pass filtering. The image is smoothed and the color blur is hiconspwuous. ^^^l^^^^ 
portions are inconveniently vague. As it is apparently understood from the above result of calculat»n. the applKafon 
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of the median filter to the edge pixel avoids blur of the edge portion. 

[0079] On the other hand, referring to Fig. 16. a color blur occurs at a pixel where the value of the color difference 
component CI is "40". It is understood that if the 3 X 3 pixel median filter is applied to this color blur pixel, as the value 
of the seven color difference components of the eight peripheral pixels is -100" and the value of the color difference 
component of the only one pixel is MO", the value of the color difference component of the color blur pixel is replaced 
with 'lOO*, thus the color blur is reduced. In this manner, the median filter also has an advantage to reduce a color 
blur while avoiding blurring an edge portion. For this reason, it is detennined whether or not the color blur pixel is an 
edge pixel at step S1 50. ^ - 

[0080] Note that the median filter has a 3 X 3 pixel size, however, if a 5 x 5 pixel median filter is used m the present 
embodiment, similarly to the case of the 3 X 3 pixel median filter, the central value can be selected from the total 
twenty-five pixels. Further, only the color difference components CI have been described, however, the color difference 
components C2 may be similarly treated. 

[0081] The calculation speed using the median filter is relatively lowerthan that using the smoothing filter. Accordingly, 
it is preferable that the use of median filter is avoided as much as possible. In a case where the expressions (9) and 
( 1 0) are employed as the edge-pixel determination references, one of these expressions Is satisfied on some occasions, 
and both expressions are satisfied on some occasions. If one of the expression is satisfied, rt is not necessary to use 
the median filter with respect to the both color difference components. Further, to avoid reduction of processing speed, 
the median filter is used only with respect to the color difference component which satisfies the expression at step 
S 1 54 while the smoothing filter is used with respect to the other color difference component at step SI 52. This entirely 
improves the processing speed. Note that in a case where the expression (11) is employed as the. edge-pixel deter- 
mination reference, determination is not made as to whether the change rates of the color difference values CI and 
C2 are high In this case, the median filter is applied to both color difference components CI ad C2 at step SI 54. 
[0082] As described above, the reduction of color blur is smoothing the image portion to be inconspicuous, which 
may result in a vague image portion. Accordingly, in the present embodiment, edge enhancement processing is per- 
formed at step 8160. ^, • 
[0083] The edge enhancement processing is performed so as to obtain the relation between unenhanced luminance 
Y of each pixel and enhanced luminance Y* as follows: 

Y* = Y + (Y-Yunsharp) O^) 

Note that "Yunsharp" represents unsharp masking processing on image data of a color blur pixel. Next, unsharp mask- 
ing processing will be described. In the present embodiment, as shown in Figs. 18(a) to 18(c). three types of 3 X 3 
pixel unsharp mask. 5X5 pixel unsharp mask and 7 X 7 pixel unsharp mask are prepared, in correspondence with 
the sizes of the smoothing filters determined at steps SI 22 to 8126. For example, if the 3 X 3 pixel smoothing filter is 
selected, the 3 x 3 pixel unsharp mask is selected. Similar to the above-described smoothing filters, tn each of the 
unsharp masks, the central value is used as a weight Y(x,y) at each pixel, and used for integrating the peripheral pixels 
with weighting corresponding to the values of the cells of the mask. For example, if the 3 X 3 pixel unsharp mask is 
used, integration is performed in accordance with the following expression: 

Yunsharp= (1/1 3 6) Z (M i j xY (x+ I . y+ j) ) 

I.J 

• (1 3) 

In the expression (13), "1 36" is a total value of weighting coefficients. This value is the total value of the cell values in 
the respective smoothing filtets of the different sizes. That is. in the 5 x 5 pixel fitter, this value is -AOO"; and in the 7 
X 7 pixel filter this value is "goO'. "Mij" represents a weighting coefficient sot in the unsharp mask cells. Y{x,y) 
represents a luminance component at each pixel. Further, "ij" is represented by horizontal and vertical coordinate 
values with respect to the unsharp mask. .... 

[00841 As •Yunsharp(x,y)- is obtained by adding the values of peripheral pixels with lowered weights with respect to 
a color blur pixel, it is also equivalent to low-pass filtered result. AccordingV, ■Y{x.y)-Yunsharp(x,y)" is equivalent to 
subtraction of lowfrequency components from all the components, i.e.. high-pass filtering. Then, the high-pass filtered 
high frequency components are added to "YCx.y)". i.e.. the high frequency components are increased. Thus, the edge 
is enhanced and the image sharpness is improved. .. ....«> •. u.^„„^ 

[0085] Note that as it is apparent from Figs. 18(a) to 18(c). In the unsharp masks, the weight coefficient becomes 
the greatest at the central portkjn. and gradually decreases towards the periphery. Accordingly, the weighting coeffl- 
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cients at the periphery have small influence on the total of the cell values. On the other hand. In the matrix calculation 
represented by the expression (1 3) or the like, as multiplication and addition are required tor the number of cells of the 
used unsharp mask with respect to the peripheral pixels of a pixel of interest, the amount of calculation may be con- 
siderably great. Accordingly, in the present embodiment, the amount of calculation is reduced and the processing 
5 speed is increased by using. 3X3 pixel, 5X5 pixel and 7 x 7 pixel unsharp masks where peripheral weighting 
coefficients are omitted. 

[0086] In the present embodiment, the size of the smoothing filter and that of the unsharp mask are the same from 
the following reason. For example, a bitmap image as shown in Fig. 19. a hatched area is smoothing processed, then 
an area surrounded by a broken line Is unsharp masked for luminance enhancement. As described above, as luminance 
10 enhancement is compensation for sharpness lost by smoothing processing, if the luminance is enhanced even outside 
of the smoothing area, the sharpness may be unnaturally improved outside of the smoothing area. For this reason, in 
the present embodiment, the size of the snr>oothing filter and that of the unsharp mask are the same, however, the 
latter size may be smaller than the former size. 

[0087] At step S160, edge enhancement processing as described above is performed, and R'. G' and B* multilevel 
IS data are obtained based on the equations (6) to (8) using the enhanced luminance Y' and the processed cotor difference 

components. That is. the cok>r blur is reduced at steps SI 52 and SI 54. and multilevel data is generated at step S160. 

The hardware construction and software to execute the above processings construct the image processing unit. 

[0088] Note that at step S1 60, occasionally the R\ G' and B' multilevel data have negative values, or the values are 

over "255-, due to use of the enhanced luminance Y', However, as the multilevel width ranges from "O" to "255". negative 
20 multilevel data are uniformly changed to "0", and multilevel data over "255" are unifonnly changed to -255". At the next 

step SI 70, the pixel of interest is moved, and similar processing Is repeated until It is determined at step 81 80 that all 

the pixels have been processed. 

[0089] In a case where the smoothing filter is applied to pixels with a color blur pixel as the central pixel and then 
the edge enhancement processing is performed, the cotor blur is reduced by the smoothing processing, and the image 
25 sharpness lost by the smoothing processing Is compensated by the edge enhancement processing. Accordingly, the 
above processings may be uniformly performed on all the cokDr blur pixels whether the cotor blur pixel is an edge pixel 
or not and similar advantages can be attained by this arrangement. In the above construction, empirically, as the edge 
portion is occasionally blurred. It Is arranged such that the median filter is used if the cotor blur pixel is an edge pixel, 
to obtain excellent result. 

30 [0090] Next, the advantages of the smoothing processing and the edge enhancement processing of the present 
embodiment will be described with a simple one-dimensional simulation model. 

[0091] Fig 20 is a table showing various parameter values in a one-dimensional simulation model in a case where 
edge enhancement processing is not perfomied. in Fig. 20, the left end R, G and 8 matrix corresponds to the cotor 
filter arrangement of a CCD. "IN " on the right side of the matrix indicates light incidence state. If the value is • 1 (multilevel 
3S data value is "255')-, it indicates iight-frradtated state. I.e.. "white-. If the value is "0 (the value of multilevel data value 
is -0")', it indicates "black". At each CCD pixel, a cotor signal which cannot be directly obtained is obtained from a color 
signal of an adjacent pixel by linear interpolation calculation. 

[0092] Initially, the simulatton model should be two-dimensional ly arranged, however, the central vertical array in the 
above RGB matrix is taken out for convenience of explanatton. In this meaning, the anay is referred to as a one- 

40 dimensional simulation model. From the above-described principle, false color signals (G(B) = 0.5 and R = 0.5) occur 
around the border between the white and black areas. The pixels where the false color signals have occurred are 
detected as color blur pixels. As the color blur pixel has been detected, the cotor difference components C1 and C2 
are calculated in accordance with the equations (6) to (8). then the color difference components C1 and C2 are smooth- 
ing processed and color difference components CV and C2' are obtained. Note that in this example, the values 1,1 

45 1111 r are used as a oneKJimensional smoothing filter. That is, the values of three pbcels abov^ 

interest and the values of three pixels below the pixel of Interest are added up and divtoed by V. Then me obt^.ned 
color difference signals CV and C2' are returned to the Initial cotor signals based on the equations (6) to (8). and 
smoothing processed R\ G* B' are obtained. As it is apparent from the figure. "(R'. G*, B')" at pixels around the border 
between the white and black areas are respectively. -(0.75. 0.61. 0.61)- and «(0.25. 0.11. 0.11)". closer to the initial 

so 'IN' values. Thus, the color blur is reduced. 

[00931 Further. Fig. 21 is a table showing various parameter values in the one-dimensional simulation modeMn a 
case where smoothing processing is performed and edge enhancement processing is performed using the unsharp 
mask of the same size of the smoothing filter. In this case, similariy to the above case, the cotor difference components 
CI and C2 are obtained and the smoothing processing is performed. When the cotor difference components are re- 

55 turned to the initial cotor signals by using the equattons (6) to (8). a oneOimensional unsharp mask »^^ving weighting 
coefficients M, 6. 18, 25, 18, 6. 1 - is utilized to use the enhanced luminance signal r. Then. (R . G , B) at pixe^ 
around the border between the white and black areas are respectively "(0.81 . 0.67, 0.67)- and -(0.10. "O O^. 'O 0^) ; 
As a value allowable as RGB signal value ranges from "0" to -r. the latter data values actually become (0.10. 0. 0) 
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■ '. In comparison with Fig. 20, the color blur is further reduced. 
[0094] Further. Fig. 22 is a table showing various pararneter values in the on-dimensional simulation model in a case 
where edge enhancement processing is similarly performed using an unsharp masit of a size smaller than, that of the 
smoothing filter. I n this case, a one-dimensional five-digit '1 . 11 . 25. 11 . 1 " unsharp mask, smaller than the smoothing 
s filter size (seven digits), is used. Calculation is performed similarly to the above processing, and -(R', G'. B')" at pixels 
around the border between the white and black areas are respectively '(0.79, 0.65, 0.65)" and "(0 10, -0.04, -0.04)". 
The color blur is not sufficiently reduced in comparison with Fig. 21 , however, it is further reduced in comparison with 
Fig. 20. 

[0095] Next, the operation of the present embodiment having the above construction will be descnbed. 

10 [009$] A case where an image obtained by the digital still camera 1 2 having a single-plate CCD is inputted into the 
computer 21 , and printing is performed by the printer 31 is considered. First, while the operating system 21a is running 
in the computer 21 , the image processing application 21 d is started, and image data is read from the digital still camera 
12 When the image data has been read by the image processing application 21 d via the operating system 21a. an 
index value for determination of the image size based on the equation (1) is obtained at step S110. The index value 

15 is compared with a predetermined threshold value, and the size of the smoothing filter and that of the unsharp mask 
used at steps S122 to S126 are determined. The determined sizes are stored in the work area. 
[0097] At step SI 30. the pixel of interest is set to an initial position, and the ARB value between the pixel of interest 
and a peripheral pixel is calculated in accordance with the equatton (2). Thereafter, at step SI 40. the change rate of 
the ARB value is examined between the pixel of interest and its peripheral pixel based on the expression (5). If the 

20 change rate is higher than the predetermined threshoW value Th. it is determined that the pixel of interest is a color 

blur pixel. , „^ . „_ 

[0098] If it is determined that the pixel of interest is a cotor blur pixel, the color difference components G1 and C2 
are calculated based on the equations (6) to (8) between the pixel of interest and its peripheral pixel at step S15Q. 
Then it is determined whether or not the pixel of interest is an edge pixel, in accordance with the determination reference 
ss such as the expression (10) or (11). If it is determined that the color blur pixel is not an edge pixel, a smoothing filter 
corresponding to the filter size, stored at steps SI 22 to S126. is employed at step SI 52. That is. with respect to the 
color difference components CI and C2. the color difference coniponents at pixels corresponding to the cells of the 
smoothing filter are added up and divided by the number of cells, so that the smoothing processed cotor difference 
components CV and C2' are obtained. . , . ^ ■ , ^ = c 

30 [00991 On the other hand. H it is determined at step S150 that the color blur pixel is an edge pixel, the 5 X 5 pixel 
median filter is applied to pixels with the color blur pixel as the central pixel at step S154. That is. the color difference 
component values of the total tw/enty-five pixels with the cotor blur pixel as the central pixel are sorted in ascending or 
descending order, then the central value is selected and replaced with the cotor difference component value of the 
color blur pixel. At this time, if the determination references of the expressions (9) and (1 0) are emptoyed at step 81 50. 
35 the median filter is used with respect to any of the color difference components CI or C2 which satisfies the expression 
(9) or (10) at step S154. white the above-described smoothing fitter is used with respect to the color difference com- 
ponent which does not satisfy the expresston (9) or (10). atstepS152. If both color difference components satisfy both 
expressions (9) and (10). the median filter is used with respect to the color difference components CI and C2 at step 
S154. On the other hand, if the determination reference of the expression (11) is emptoyed at step S150. the median 
40 filter is used with respect to both cotor difference components CI and C2 at step SI 54. . v ^ 

rOIOOl Thereafter, at step S160. the low frequency components Yunsharp of the luminance components Y are ob- 
tained with respect to the pixel of interest and its peripheral pixels by using the urisharp mask of the size detennined 
at steps SI 22 to SI 26. and further, the high frequency components are obtained by subtracting the low <fequency 
components Yunsharp from the initial luminance components Y Then, the high frequency components added o 
the initial luminance components Y to obtain the enhanced luminance components Y'. M«xe «P«'=J^,5;, ^ 
perfom^ed based on the expression (13) or the like. Then, the smoothing processed R and B ",uitnevel^^^ 
obtained based on the equations (6) to (8) using the enhanced luminance components Y and he P;«;^ssed ^lo^ 
difference components. The above processing is repeated while the pixel of interest is moved until rt is determined at 

step SI 80 that all the pixels have been processed. ^ ri«nia« via the 

[0101] When all the pixels have been processed, the image processed data is displayed on the display 32 via the 
display driver 21c. If the display is excellent, the data is printK^utputted by the printer 31 via the printer dnver 21^at 
is the printer driver 21b inputs the color-blur reduced RGB multilevel data, and performs 'f'^'^^''°'\'^"^^f^r^f 
o the printhead area of the printer 31 through predetemiined resolution converston. Then, the printer dr^^er 21b color- 
Sivefs me rasterized data from RGB to CMYK data, and further, convens the CMYK multilevel data into binary data 
55 and outDUts the data to the printer 31 . . .. . 

[0102 ' By he above processing, the image data obtained by the digital still came« 12 is f^^-^^'^!;;^^^^^;^ 
essedsuchthatonlyacolorbluratacotorblurportionisreduced.thendisplayedonthed«pay32.^^^^ 

by the printer 31 . That is. as image processing is performed so as to reduce the cotor blur at the color blur portion, the 



45 



SO 



13 



EP 0 920 221 A2 

amount of calculation is reduced and high-speed image processing is realized. 

[0103] In this manner, the computer 21 as the nucleus of the image processing determines at steps S130 and S140 
whether or not the pixel of interest is a color blur pixel based on low-density element color intensity, and if the computer 
21 determines that the pixel of interest is a color blur pixel, it determines at step SI 50 whether or not the color blur 
s pixel is an edge pixel. If the color blur pixel is not an edge pixel, the computer 21 performs image processing by using 
the smoothing filter on the pixel at step ST52, while if the color blur pixel is an edge pixel, the computer 21 performs 
image processing by using the median filter at step S1 54. so as to reduce the color blur Thus, the amount of calculation 
is reduced, and high-speed image processing is realized. 

[0104] The above-described embodiment is realized by software processing in the computer system 10. However. 

10 the present invention can be easily realized by hardware circuits. In this case, the present invention is realized by a 
memory for reading image data, IC circuits or the like for realizing the processings at the respective steps in the flowchart 
of Fig. 6. One of the IC circuits is a color-blur pixel detection circuit for accessing the image memory to scan the position 
of a pixel of interest at S170 and SI 80, and detecting existence/absence of color blur at each pixel of interest; another 
one of the iC circuits is a color-blur reduction processing circuit for accessing peripheral pixels around a pixel of interest, 

IS determined to be a color blur pixel in correspondence with the processing at steps S140 to S160, as the reference, 
using the smoothing filter or median fitter to reduce the color blur, and finally performing edge enhancement. The 
respective IC circuits are accompanied by other hardware circuits. In this case, the color-blur pixel detection circuit 
corresponds to the above-described color-blur pixel detection unit, and the color-blur reduction processing circuit cor- 
responds to the above-described Image processing unit 
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Claims 



1 . An Image processing apparatus which performs image processing on image data consisting of dot-matrixed pixels. 
2S generated by obtaining image data by a single-plate solid image pickup device where a plurality of color filters of 

element color components are arranged in mosaic in a nonuniform densities and supplementing the image data 
by calculation to change the nonuniform densities to uniform densities, said apparatus comprising: 

a color-blur pixel detectbn unit which detects a color blur pixel in said image data; and 
30 an image processing unit whteh performs image processing on pixels within a predetermined range having 

said detected color blur pixel as a reference pixel, so as to reduce a color blur. 

2. The image processing apparatus according to claim 1 , wherein said color-blur pixel detection unit detects said 
color blur pixel based on change rate of element color intensity for a low-density color filter, between closely ad- 

3S jacent pixels. 

3. The image processing apparatus according to claim 2. wherein said cok>r-blur pixel detection unit detects said 
color blur pixel based on the change rate of difference between a reference element color intensity and the element 
color intensity for a low-density color filter, between adjacent pixels. 



4. The image processing apparatus according to claim 3, wherein if there are a plurality of low-density color filters, 
said color-blur pixel detection unit detects said color blur pixel based on the change rate of difference between 
element color intensities for the tow-density color filters between adjacent pixels. 

4S 5. The image processing apparatus according to any one of claims 2 to 4. wherein said color-blur pixel detectran unit 
detects said color blur pixel based on said change rate between adjacent low-density pixels. 

6. The image processing apparatus according to any one of claims 1 to 5. wherein sakj image processing unit performs 
smoothing processing on color difference components, obtained by subtracting luminance components from ele- 

so ment color components of the pixels within the predetermined range having said color blur pixel as the reference 

pixel, and returns the smoothing processed components to the initial element color components. 

7. The image processing apparatus according to claim 6. wherein said image processing unit perfomns edge en- 
hancement processing. 

8. The image processing apparatus according to claim 7, wherein said image processing unit performs edge en- 
hancement processing on pixels within a range subjected to the smoothing processing. 
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9. The image processing apparatus according to any one of claims 6 to 8. wherein when said image processing unit 
performs the smoothing processing on the pixels within the predetermined range having said detected color blur 
pixel as the reference pixel, if the size of a processing object image is large, said image processing unit increases 
the range subjected to the smoothing processing, while if the size of the image is small, said image. processing 

s unit reduces the range subjected to the smoothing processing. 

1 0. The image processing apparatus according to any one of claims 1 to 5, wherein said image processing unit replaces 
a color difference component having a central value of color difference components, obtained by subtracting lu- 
minance components from element color components of the pixels within the predetemiined range having said 

10 color blur pixel as the reference pixel, with a color difference component of said color blur pixel, and returns the 

color difference components to the initial element color components. 

11 The image processing apparatus according to any one of clainfis 1 to 5, wherein said image processing unit de- 
termines whether or not said color blur pixel is an edge pixel, and if said image processing unit determines that 

IS saidcolorblurpixel isan edgepixel, replacesacolordifferencecomponenthavingacentral value of color difference 

components, obtained by subtracting luminance components from element color components of the pixels within 
the predetermined range having said color blur pixel as the reference pixel, with a color difference component of 
said color blur pixel, while if said image processing unit determines that said color blur pixel is not an edge pixel, 
performs the smoothing processing on the color difference components, obtained by subtracting the luminance 

20 components from the element color cornponents of the pixels within the predetermined range having said color 

blur pixel as a reference pixel, and returns the color difference components to the initial element color components. 

12. An Image processing apparatus which performs image processing on image data consisting of dot-matrixed pixels, 
generated by obtaining image data by a single-plate solid image pickup device where a plurality of color filters of 

25 element color components are arranged in mosaic Hi nonuniform densities and supplementing the Image data by 

calculation to change the nonuniform densities to uniform densities, said apparatus compnsing: 

a memory in which said image data is stored, . wi • i 

a color-blur pixel detection circuit which accesses said memory and detects a position of a color blur pixe 
30 based on the difference between a pixel of interest and its peripheral pixel while sequentially moving the pixel 

of interest; and ^ ^ . . , , _ 

a color-blur reduction processing circuit which reads data of pixels within a predetermined range havHig the 
pixel of Interest, detected as the color blur pixel, as a reference pixel, then performs calculation to reduce a 
color blur, and updates data of the pixel of interest stored in said memory wXh calculated data. 

13. An image processing method for performing image processing on image data consisting of dot-matrixerd pixels 
generated by obtaining image data by a single-plate solid image picKup device where a plurality of color Me^ of 
element color components are arranged in mosaic in nonunlfom, densities and supplementing the image data by 
calculation to change the nonuniform densities to uniform densities, said method comprising the steps of: 

detectina a color blur pixel In said image data; and ... 
per^ornihgimagepro^ssingonpbcetswithinapredeterniinedrangehavings^^^ 

pixel so as to reduce a color blur. 
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